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United  States  General  Accounting  Office  Resources,  Community,  and 

Washington,  D.C.  20548  Economic  Development  Division 


B-283632 
June  12,  2000 

The  Honorable  Richard  C.  Shelby 
Chairman,  Subcommittee  on  Transportation 
Committee  on  Appropriations 
United  States  Senate 

Dear  Mr.  Chairman: 

Currently,  the  Federal  Aviation  Administration  (FAA)  relies  principally  on  a 
ground-based  navigation  system  that  uses  various  types  of  equipment  to 
assist  pilots  in  navigating  their  assigned  routes  and  to  provide  them  with 
guidance  for  landing  their  aircraft  safely  in  different  types  of  weather. 
However,  this  ground-based  navigation  system  is  aging  and  limited  in  its 
geographic  coverage.  FAA  is  planning  a  transition  from  its  ground-based 
navigation  system  to  a  satellite-based  system  using  radio  signals  generated 
by  the  Global  Positioning  System  (GPS)  to  provide  greater  geographic 
coverage,  among  other  things.  The  Department  of  Defense  developed  GPS 
to  support  military  missions  and  functions.  However,  the  system  is  now  a 
dual-use  system,  and  other  users — pilots,  truckers,  and  boaters — rely  on 
signals  from  the  GPS  satellites  to  calculate  their  time,  speed,  and  position 
anywhere  on  or  above  the  earth’s  surface.  As  part  of  its  efforts  to  maintain 
GPS  and  make  it  more  useful  for  civilians,  in  May  2000,  Defense  ceased  its 
practice  of  intentionally  degrading  the  accuracy  of  the  GPS  signal  available 
for  civil  use.1  Furthermore,  Defense  plans  to  begin  gradually  replacing  the 
existing  satellites  with  new  ones  that  will  also  improve  system 
performance. 

Although  GPS  already  provides  some  critical  information  to  pilots,  FAA 
believes  that  even  with  the  greater  accuracy  and  other  improvements 
expected  from  Defenses  newer  satellites,  this  system  will  not  satisfy  all 
civil  aviation  requirements  for  ensuring  safe  aircraft  operations.  For 
example,  FAA  requires  that  its  navigation  system  be  unavailable  no  more 
than  5  minutes  per  year  for  some  types  of  navigation  and  landing.  In 
contrast,  GPS  could  be  unavailable  for  periods  of  time  equaling  up  to  4  days 
per  year.  To  satisfy  its  requirements,  FAA  decided  in  the  1980s  to  augment 


1  In  the  past,  the  Department  degraded  the  accuracy  of  the  GPS  signal  using  a  process 
known  as  “selective  availability." 
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Results  in  Brief 


GPS  with  other  navigational  aids — the  Wide  Area  Augmentation  System 
(WAAS)  and  the  Local  Area  Augmentation  System.  These  systems  consist 
primarily  of  a  network  of  ground  stations  that  receive,  process,  and 
validate  data  from  GPS  before  transmitting  these  data  to  pilots. 

In  deciding  to  augment  GPS,  FAA  has  conducted  several  benefit-cost 
analyses — most  recently  in  1999.  In  its  latest  analysis,  FAA  estimated  that 
its  future  investment  in  this  augmented  navigation  system  could  exceed  $8 
billion  from  2000  through  2020. 2  WAAS  is  the  largest  component  of  this 
augmented  system — 37  percent  of  the  total.  Over  the  years,  the  cost  of 
developing  WAAS  has  increased  by  over  $500  million  primarily  because  of 
unanticipated  development  and  additional  program  support  costs.  In 
addition,  WAAS  has  been  delayed  for  over  3  years  and  has  experienced 
performance  problems. 

In  light  of  the  expected  cost  of  this  new  navigation  system  and  continuing 
concerns  about  WAAS’  ability  to  achieve  cost,  schedule,  and  performance 
goals,  you  asked  us  to  provide  information  on  whether  (1)  the  Department 
of  Defenses  current  GPS  or  its  planned  improvements  for  GPS  can  meet 
FAAs  navigation  requirements,  (2)  the  benefits  of  FAA’s  chosen  approach  to 
an  augmented  system  currently  outweigh  the  cost  of  this  system,  and  (3) 
other  technologies  are  available  to  meet  FAA’s  requirements  and  users’ 
needs  for  a  new  navigation  system.  In  conducting  this  review,  we  examined 
studies  and  spoke  with  experts  in  aviation  navigation  and  related 
technologies  to  obtain  their  views  on  the  capability  of  FAAs  new  navigation 
system  and  alternatives  to  that  system.  Appendix  I  discusses  our  detailed 
scope  and  methodology.  Appendix  II  lists  the  experts  we  spoke  with,  some 
of  whom  represent  various  technologies. 


According  to  the  studies  we  reviewed  and  experts  we  contacted,  the 
current  Global  Positioning  System  does  not  meet  all  of  FAAs  civil  aviation 
navigation  requirements  for  accuracy,  integrity,  and  availability.  FAA 
defines  accuracy  as  the  degree  to  which  a  navigation  system  calculates  an 


2  Throughout  this  report,  we  refer  to  the  augmented  satellite  navigation  system  and  its 
components — including  the  Local  Area  Augmentation  System  and  existing  ground-based 
navigation  aids — as  FAA’s  new  navigation  system.  This  system  also  provides  guidance  to 
help  pilots  land  at  airports.  Also,  throughout  this  report,  unless  otherwise  specified,  costs 
are  presented  in  “then-year-dollars,"  which  are  current  dollars,  inflated  using  Office  of 
Management  and  Budget  guidance. 
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aircrafts  true  position.  Integrity  is  the  ability  of  a  navigation  system  to 
provide  timely  warnings  when  its  signal  is  providing  misleading 
information  that  could  potentially  create  hazards  for  pilots  and  thus  should 
not  be  used.  Availability  is  the  probability  that  a  navigation  system  meets 
the  accuracy  and  integrity  requirements.  Even  though  the  Department  of 
Defense  has  made  the  current  GPS  signal  provided  for  civilian  use  more 
accurate  and  plans  to  further  improve  GPS  by,  for  example,  providing  an 
additional,  higher-powered  signal  to  combat  interference,  GPS  still  will  not 
fully  meet  FAA’s  requirements  for  navigation  and  landing.  This  is  because 
GPS  does  not  provide  the  assurance  that  its  signal  will  be  available  virtually 
all  the  time. 

FAA’s  1999  analysis  concluded  that  the  quantified  benefits  of  its  approach 
would  outweigh  the  cost.  Since  completing  this  analysis,  FAA  has 
experienced  delays  and  cost  increases  primarily  because  of  difficulties  in 
meeting  its  integrity  requirement.  As  a  result,  it  is  unclear  whether 
quantified  benefits  will  still  outweigh  cost.  For  example,  to  meetFAA’s 
integrity  requirement — which  requires  the  Wide  Area  Augmentation  System 
to  virtually  never  fail  to  warn  pilots  of  potentially  hazardously  misleading 
information — we  estimate  that  the  agency  may  need  3  or  more  years,  at  an 
additional  cost  of  between  $200  million  to  $240  million,  to  demonstrate  that 
this  requirement  can  be  met.  The  system,  with  the  ability  to  provide 
integrity,  was  to  have  been  operational  by  September  2000.  We  are  making 
recommendations  to  better  ensure  that  FAA  delivers  its  new  navigation 
system,  on  time  and  within  budget,  and  that  it  meets  performance 
requirements.  Transportation  and  Defense  officials  as  well  as  the  officials 
from  the  Satellite  Navigation  User  Group — representing  commercial, 
general  avaition,  and  Defense  users — acknowledged  the  problems 
encountered  in  developing  the  new  navigation  system.  Transportation 
officials  concurred  with  GAO’s  conclusions  and  recommendations. 

At  the  present  time,  no  other  navigation  technologies — including  variations 
of  ground-based  and  less  robust  satellite-based  systems — are  available  to 
meet  FAA’s  requirements  and  users’  needs  for  precise  landing  guidance  at 
more  airports.  The  Wide  Area  Augmentation  System  is  designed  to  provide 
such  guidance  to  improve  safety  and  offer  access  to  more  airports. 
However,  as  noted,  this  system  is  experiencing  difficulties  that  cast  doubt 
on  whether  it  will  perform  as  designed  at  a  reasonable  cost  and  be 
delivered  on  a  reasonable  schedule.  Moreover,  experts  in  alternative 
navigation  technologies,  some  of  which  compete  with  the  Wide  Area 
Augmentation  System,  told  us  that  users  may  have  overstated  their  need 
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for  precise  landing  guidance  and  that  other  navigation  technologies  could 
satisfy  most  of  these  needs. 


Background 


Currently,  civil  aviation  relies  principally  on  a  ground-based  navigation 
system  that  uses  various  types  of  equipment  to  provide  navigation  and 
landing  services  to  pilots  in  different  types  of  weather.  This  equipment 
meets  FAA’s  performance  requirements  for  accuracy,  availability,  and 
integrity;  however,  it  is  aging  and  has  limitations  in  its  geographic  coverage. 
For  example,  under  today’s  ground-based  navigation  system,  pilots  fly 
structured  routes  (referred  to  as  highways-in-the-sky)  that  may  not  be  the 
most  direct  and  fuel-efficient.  The  wider  coverage  provided  by  WAAS, 
coupled  with  other  improvements,  would  not  restrict  pilots  to  these 
structured  routes  and  would  therefore  result  in  more  direct  and  fuel- 
efficient  flights. 

To  a  lesser  degree,  civil  aviation  relies  on  GPS  for  its  navigation  needs.  The 
Department  of  Defense  (DOD)  maintains  a  constellation  of  24  orbiting  GPS 
satellites  for  both  military  and  civilian  use.3  These  satellites  are  positioned 
so  that  at  any  given  time  the  signals  from  a  minimum  of  four  satellites  will 
be  available  to  users. 

FAA  is  developing  a  new  navigation  system  to  augment  GPS  in  order  to 
provide  broader  geographic  coverage,  among  other  things.  The  largest 
component  of  this  system  is  WAAS.  When  fully  developed,  WAAS  could 
comprise  a  network  of  up  to  76  ground  stations  and  three  to  four 
geostationary  communications  satellites,  (see  fig.  1.)  The  WAAS  network  is 
being  designed  to  provide  the  same  level  of  service  as  today’s  ground-based 
equipment  and  is  expected  to  support  navigation  through  all  phases  of 
flight  as  well  as  nonprecision  and  category  I  precision  landing  approaches 
for  a  wider  geographic  area.4  In  a  nonprecision  approach,  the  pilot  relies  on 
instruments  on  board  the  aircraft  to  guide  it  safely  from  a  height  ranging 
from  between  700  and  400  feet  above  touchdown.  In  contrast,  in  a  category 
I  precision  approach,  the  pilot  relies  on  instruments  to  provide  an  aircraft 
with  safe  vertical  guidance  to  a  height  of  not  less  than  200  feet  above 
touchdown.  Currently,  pilots  can  make  precision  approaches  at  only  about 


3  Replacement  satellites  will  be  launched  as  needed. 

4  Navigation  guidance  is  provided  to  pilots  through  all  phases  of  flight — at  high  altitudes  and 
in  areas  close  to  airports. 
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625  airports.  WithWAAS,FAA  estimates  that  it  may  be  possible  to  expand 
this  capability  to  approximately  3,300  airports,  thus  providing  benefits  to 
more  users.  To  obtain  the  full  benefit  from  WAAS,  it  would  be  necessary  for 
these  airports  to  incur  costs  for  items  they  would  need  for  providing 
greater  access  at  200  feet,  such  as  airport  lighting  systems,  which  could 
cost  between  $1  million  to  $2  million  per  airport  system.  Additionally,  many 
pilots  are  not  now  qualified  to  fly  these  precision  approaches  and  will 
therefore  need  training  to  obtain  the  full  benefits  ofWAAS. 
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Figure  1 :  The  Operating  System  forWAAS 
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As  part  of  its  augmentation  of  GPS,  FAA,  in  partnership  with  industry,5  is 
also  developing  the  Local  Area  Augmentation  System  (LAAS)  to  support, 
among  other  things,  even  more  stringent  precision  approach  guidance  than 
expected  from  WAAS.  For  example,  in  these  approaches,  LAAS  is  expected 
to  provide  pilots  with  safe  vertical  instrument  guidance  to  heights  ranging 
from  less  than  200  feet  to  down  to  the  runway  surface.  While  LAAS  is 
independent  of  WAAS,  it  is  also  expected  to  complement  WAAS  and 
provide  precision  approaches  at  airports  where  WAAS  does  not  provide 
sufficient  geographic  coverage.  LAAS  will  require  the  development,  testing, 
evaluation,  and  fielding  of  a  new  generation  of  ground  stations — up  to  160. 
In  January  1998,  FAA  approved  development  costs  and  schedules  for  LAAS. 
At  that  time,  FAA  estimated  that  LAAS  would  cost  $530  million  and  would 
be  operational  by  2003. 

Table  1  shows  the  history  of  development  costs  and  schedules  for  WAAS 
and  LAAS. 


Table  1:  Development  Costs  and  Schedules  for  WAAS  and  LAAS,  1994  Through 
September  1999 


Dollars  in  millions 

As  of 

1994 

Jan.  1998 

Jan. 1999 

Sept.  1999 

Types  of  costs 

Total  WAAS 

development 

costs 

$508 

$1,007a 

$1,007 

$2,484b 

Total  LAAS 

development 

costs 

C 

$530 

$530 

$720d 

Schedule 

Information 

WAAS’  initial 
capability 

June  1997 

July  1999 

Sept.  2000 

Sept.  2000 

5  Beginning  is  fiscal  year  1999,  FAA  established  a  partnership  with  interested  commercial 
entities  for  the  purpose  of  developing  LAAS.  FAA  expects  this  partnership  to  culminate  in 
the  development  of  a  certified  category  I  precision  approach  using  LAAS  by  the  end  of  fiscal 
year  2002. 
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(Continued  From  Previous  Page) 


As  of 

1994 

Jan.  1998  Jan.  1999 

Sept.  1999 

Schedule 

Information 

WAAS’  full  Dec.  2000 

capability 

Dec.  2001 

To  be 

determined 

Dec.  2006 

LAAS’  first  site  c 
implementation 

2003 

2003 

2003 

LAAS’  last  site  c 
implementation 

2006 

2006 

2010 

Note:  Since  1996,  FAA  has  been  including  life-cycle  costs,  which  include  costs  for  developing, 
operating,  and  maintaining  projects.  The  current  life-cycle  cost  estimate  for  WAAS  is  $3,187.6  million. 


a  The  Jan.  1998  program  development  costs  for  WAAS  include  the  prime  contractor  costs, 
development  of  standards  and  procedures,  technical  engineering  and  program  support,  and  the  first 
year  of  costs  for  satellites. 

b  The  Sept.  1999  estimate  for  WAAS  development  costs  includes  $1.3  billion  in  satellite  service 
acquisitions  through  2020.  In  earlier  estimates,  satellite  service  acquisition  costs  were  included  in  the 
cost  of  operating  WAAS. 

c  Costs  and  schedules  were  not  developed  until  1998. 

d  The  Sept.  1 999  LAAS  cost  increase  is  due  in  part  to  FAA  now  planning  to  acquire  up  to  1 60  systems. 
In  earlier  cost  estimates,  FAA  only  planned  to  acquire  143  systems. 

Source:  FAA. 


FAA  is  retaining  about  30  percent  of  its  ground-based  navigation 
infrastructure  to  address  concerns  about  the  vulnerability  of  the  GPS 
signal,  which  WAAS  relies  on,  and  to  support  those  users  who  choose  not 
to  purchase  the  equipment  that  must  be  used  with  WAAS.  This 
infrastructure,  along  with  WAAS  and  LAAS,  make  up  the  components  of 
FAA’s  new  navigation  system.  Both  WAAS  and  LAAS  would  require  airlines 
and  general  aviation  users  to  purchase  on-board  equipment  for  receiving 
signals  from  this  new  technology.  These  purchases  are  expected  to  occur 
over  time,  as  the  new  navigation  system  is  developed. 

WAAS  and  LAAS  are  being  developed  under  a  single  FAA  integrated 
product  team,  which  includes  representatives  from  FAA’s  aircraft 
certification  and  acquisition  organizations.  FAA  established  cross¬ 
functional  teams  to  help  ensure  that  systems  are  developed  and 
implemented  in  an  efficient  and  effective  manner.  These  teams  are  to  be 
empowered  to  make  decisions  affecting  systems  and  services,  from  their 
inception  to  their  eventual  disposal  or  termination.  The  effective  operation 
of  the  integrated  teams  is  key  to  FAA’s  goal  of  producing  timely,  cost- 
effective  acquisitions. 
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GPS  Needs 

Augmentation  to  Meet 
FAA’s  Civil  Aviation 
Navigation 
Requirements 


Studies  we  reviewed,  experts  we  contacted,  and  DOD  officials  we 
interviewed,  agreed  that  to  meet  FAA’s  civil  aviation  requirements,  FAA 
must  augment  even  the  improved  GPS.  For  example,  according  to  a  1999 
Johns  Hopkins  University  study,6  augmentation  was  needed  because  GPS 
failed  to  meet  the  critical  requirement  of  system  availability,  which  is  the 
probability  that  at  any  given  time  a  navigation  system  will  meet  the 
accuracy  and  integrity  requirements  for  a  specific  phase  of  flight.7  FAA 
defines  accuracy  as  the  degree  to  which  an  aircraft’s  position,  as  calculated 
by  its  navigation  system,  conforms  to  its  true  position.  Integrity  is  the 
ability  of  a  navigation  system  to  provide  timely  warnings  when  its  signal  is 
providing  misleading  information  that  could  potentially  create  hazards  for 
pilots  and  thus  should  not  be  used  for  navigation.  This  study  considered  the 
impact  of  DOD’s  decision  to  stop  intentionally  degrading  the  accuracy  of 
the  signal  for  civilian  use  and  concluded  that  augmentation  was  still 
necessary.  In  May  2000,  DOD  actually  stopped  degrading  its  signal  for 
civilian  use.  As  a  result,  the  predicted  accuracy  for  civilian  use  will  increase 
from  about  100  meters  (about  300  feet)  to  20  meters  (about  60  feet). 
However,  FAA  requires  an  accuracy  of  about  7.6  meters  (about  23  feet)  for 
landing  aircraft. 


6  This  study,  entitled  GPS  Risk  Assessment  Study,  Final  Report  (Jan.  1999),  was  conducted 
by  the  Johns  Hopkins  University  Applied  Physics  Laboratoiy  and  was  co-sponsored  by  FAA, 
the  Air  Transport  Association,  and  the  Aircraft  Owners  and  Pilots  Association.  According  to 
the  Air  Transport  Association,  the  primary  purpose  of  this  study  was  to  determine  the 
capability  of  the  augmented  GPS  signal  using  WAAS  and  LAAS. 

7  Continuity  and  service  volume  are  also  considered  major  requirements,  and  they  are 
derived  from  the  accuracy,  integrity,  and  availability  requirements.  Continuity  is  the 
probability  that  a  navigation  signal  will  meet  accuracy  and  integrity  requirements 
continuously  for  a  specified  period.  Service  volume  is  the  area  of  coverage  for  which  a 
navigation  signal  will  meet  availability  requirements. 
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Moreover,  Defense  plans  to  begin  gradually  replacing  the  existing  satellites 
with  new  ones  that  will  have  an  additional  higher-powered  signal  to  help 
combat  interference  and  improve  overall  system  performance.  However, 
even  with  these  improvements,  GPS  still  falls  short  in  meeting  the 
requirement  that  it  be  available  virtually  all  of  the  time.  For  example,  the 
Johns  Hopkins  study  found  that  GPS  could  be  unavailable  for  periods  of 
time  equaling  up  to  4  days  per  year  for  en  route  navigation.8, 9  In  contrast, 
FAA  requires  that  its  navigation  system  be  unavailable  no  more  than  5 
minutes  per  year  for  en  route  navigation  and  nonprecision  approaches. 
Furthermore,  GPS  alone  does  not  provide  guidance  for  precision 
approaches. 

In  addition,  an  expert  we  spoke  with  noted  that  FAA’s  efforts  to  augment 
GPS  are  in  line  with  those  of  other  nations  to  augment  a  navigation  system 
based  on  global  satellites.10  Nations  with  navigation  systems  based  on 
global  satellites  will  likely  augment  the  signal  provided  by  the  global 
system  in  order  to  maintain  surveillance  over  civil  navigation  services.  This 
would  enable  them  to  independently  monitor — from  the  ground — the 
usability  of  the  GPS  signals  and  provide  GPS  status  for  their  air  traffic 
controllers  and  aircraft. 


8  En  route  navigation  occurs  when  planes  are  in  transit  over  the  continental  United  States. 

9  According  to  DOD,  GPS’  availability  is  contingent  upon  the  alignment  of  satellites  with  the 
earth’s  rotation. 

10  Europe  and  Japan  are  currently  developing  augmentation  systems. 
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Integrity  Issues  and 
Other  Problems  in 
Developing  WAAS 
Raise  Questions  About 
Whether  the  Quantified 
Benefits  of  FAA’s  New 
Navigation  System 
Outweigh  Its  Cost 


According  to  FAA’s  1999  analysis,  its  new  navigation  system  yielded 
quantified  benefits  that  exceeded  costs  over  the  period  2000  through  2020, 
but  development  problems — principally  related  to  proving  the  WAAS 
integrity  requirements — will  raise  costs,  making  it  unclear  whether 
quantified  benefits  still  exceed  the  cost  of  the  new  system.  Also,  in  the 
short  term,  because  of  the  delays  resulting  from  these  problems,  users  will 
not  receive  the  benefits  they  expected  from  the  system  later  this  year,  such 
as  greater  capability  to  land  at  more  airports  in  bad  weather.  Additional 
costs  are  occurring  because  FAA  has  not  appropriately  overseen  its 
contractor11  and  has  underestimated  software  costs.  Finally,  it  has  been 
suggested  that  the  system’s  cost  may  be  higher  if  performance  problems 
indicate  that  FAA  needs  to  maintain  separate  navigation  equipment  at  the 
airport. 


11  The  Raytheon  Company  has  been  the  prime  contractor  since  May  1996. 
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FAA’s  Chosen  Alternative 
Yielded  Net  Benefits,  but 
Problems  Will  Most  Likely 
Reduce  These  Benefits 


In  1999,  in  response  to  a  congressional  request,  FAA  reevaluated  its  plans 
for  a  new  navigation  system.12  As  part  of  this  reevaluation,  FAA  conducted 
a  benefit-cost  analysis  of  four  alternatives.  According  to  FAA’s  analysis, 
continuing  with  the  WAAS/LAAS  system,  coupled  with  using  about  30 
percent  of  the  agency’s  current  ground-based  infrastructure  as  a  backup  to 
WAAS,  would  yield  the  greatest  net  quantified  benefits  (dollar  value  of 
benefits  minus  costs),13  regardless  of  whether  passengers’  time  savings  are 
included.14, 15  According  to  FAA,  for  2000  through  2020,  there  is  an  80- 
percent  chance  that  the  net  benefits  of  its  preferred  alternative  are  about 
$2.5  billion  if  passengers’  time  savings  are  counted  and  $72  million  if  these 
savings  are  not  counted.  Expressed  another  way,  the  benefit-cost  ratios  for 
these  two  scenarios  are  2.4  and  1.1,  respectively.16  The  benefit-cost  analysis 
also  identified  at  least  1 1  benefits  that  were  not  quantified.  Included  among 
these  were  safety  benefits,  such  as  improved  surface  surveillance,  as  well 
as  operational  benefits,  such  as  enabling  more  landings  at  airports  whose 
operations  are  today  limited  by  the  lack  of  ground-based  navigational 
aids.17  According  to  FAA,  its  decision  to  pursue  this  approach  had  the 
support  of  major  users.18  (See  app.  Ill  for  a  discussion  of  FAA’s  analysis.) 


12  See  Satellite  Navigation  Investment  Analysis  Report ,  Federal  Aviation  Administration 
(Sept.  25,  1999). 

13  As  an  alternative  to  the  benefit-cost  ratio,  for  which  the  present  value  of  benefits  is 
divided  by  the  present  value  of  costs,  analysts  sometimes  calculate  the  present  value  of  net 
benefits.  This  value  is  equal  to  the  present  value  of  benefits  minus  the  present  value  of 
costs. 

14  DOT  values  passenger  time  from  $22  to  $33  per  hour,  depending  on  the  nature  of  air  travel. 
In  1998,  our  review  of  the  economic  literature  found  that  no  consensus  exists  on  the  validity 
of  valuing  small  increments  of  time,  in  part  because  passengers  might  not  perceive  and 
value  time  savings  of  as  little  as  30  seconds. 

15  It  should  be  noted  that  the  other  three  options  had  benefit-cost  ratios  of  less  than  1. 

1CA  benefit-cost  ratio  is  a  measure  of  the  relationship  between  the  present  value  of  a 
projects  benefits  and  costs.  Since  benefits  are  divided  by  costs,  any  ratio  above  1.0 
indicates  that  the  project  is  cost-beneficial;  any  ratio  below  1.0  indicates  that  the  project  is 
not  cost-beneficial. 

17  Data  were  not  available  to  quantify  5  of  the  1 1  benefits.  For  the  other  six,  FAA  determined 
that  the  benefits  would  derive  from  WAAS,  combined  with  other  technologies,  and  therefore 
did  not  attempt  to  separate  and  quantify  the  WAAS  benefits. 

18  The  Satellite  Navigation  User  Group — which  includes  representatives  from  commercial, 
general  aviation  and  Department  of  Defense  users — was  created  to  achieve  a  consensus 
throughout  the  user  community  and  within  FAA  for  making  the  transition  to  the  new 
navigation  system.  In  Mar.  2000,  this  group  reaffirmed  its  support  for  WAAS. 
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While  we  did  not  conduct  a  detailed  benefit-cost  analysis,  we  believe  that 
under  certain  conditions  the  quantified  net  benefits  of  FAA’s  chosen 
alternative,  without  passenger  value  of  time,  could  be  negative,  and  the 
benefit-cost  ratio  could  be  less  than  1.  This  could  occur  because  of  added 
costs  resulting  from  delays  or  other  problems.  Furthermore,  these  delays 
could  have  implications  forFAA  and  users. 


The  Requirement  for  System 
Integrity  Will  Delay  WAAS 
Deployment  and  Was  Not 
Addressed  in  a  Timely 
Manner  by  FAA 
Management  and  the 
Integrated  Product  Team 


In  December  1999,  FAA  found  that  the  WAAS  design  could  not  be  relied 
upon  to  satisfy  the  agency’s  requirement  for  system  integrity  for  precision 
approaches,  which  stipulates  that  WAAS  cannot  fail  to  warn  pilots  of 
misleading  information  that  could  potentially  create  hazardous  situations 
more  than  once  in  10  million  approaches.  Consequently,  FAA  has 
determined  that  it  will  not  make  its  scheduled  date  of  September  2000  to 
begin  providing  an  initial  capability  for  precision  guidance  (category  I 
approaches)  through  WAAS.19  The  delay  could  have  implications  forFAA, 
system  users,  and  equipment  manufacturers.  For  example,  FAA  may  need 
to  buy  new  ground-based  navigation  equipment  or  maintain  existing 
equipment  longer  than  expected — maintaining  existing  equipment  costs 
about  $170  million  annually,  according  to  FAA.  Likewise,  system  users  and 
equipment  manufacturers  could  question  the  wisdom  of  making  further 
investments  in  WAAS  technology  Because  of  these  implications,  FAA,  with 
users’  support,  has  decided  to  provide  only  a  limited  precision  guidance 
capability  with  WAAS  by  2002. 20  FAA  has  yet  to  determine  when  WAAS  will 
achieve  its  initial  capability.  However,  FAA  and  major  user  groups  contend 
that  they  still  will  receive  benefits  from  a  limited  WAAS.  Using  information 
provided  by  FAA  and  its  experts,  we  estimate  that  resolving  the  integrity 
problem  could  potentially  delay  making  WAAS’  initial  capability  available 
by  3  years  or  more  and  add  approximately  $200  million  to  $240  million  to 
the  cost  of  developing  WAAS. 


19  WAAS’  initial  capability  was  defined  as  vertical  guidance  of  200  feet  above  touchdown 
with  a  one-half  to  three-fourths  mile  visibility  and  a  19.2  meter  vertical  protection  limit  in 
which  an  aircraft  can  maneuver  and  still  land  safely.  This  capability  was  to  be  available  95 
percent  of  the  time  to  about  50  percent  of  the  continental  United  States. 

20  By  2002,  FAA  plans  to  provide  vertical  guidance  of  350  feet  above  touchdown  with  one 
mile  visibility  and  a  50  meter  vertical  protection  limit  in  which  an  aircraft  can  maneuver  and 
still  land  safely.  This  capability  is  to  be  available  95  percent  of  the  time  to  about  75  percent 
of  the  continental  United  States. 


Page  15 


GAO/RCED/AIMD-00-130  National  Airspace  System 


B  283632 


To  satisfy  the  integrity  requirement,  FAA  and  its  contractor  plan  to  make 
changes  to  improve  the  calculations  for  better  identifying  misleading 
information  that  could  potentially  create  hazards  for  pilots;  these  changes 
will  cause  software  changes  and  may  result  in  hardware  upgrades.  A  team 
consisting  of  FAA  officials,  its  contractor,  and  consultants  proposed  these 
changes  and  plans  to  be  actively  involved  in  ensuring  that  they  will  result  in 
proving  the  system’s  integrity  performance.  By  the  end  of  2000,  the  team 
expects  to  determine  what,  if  any,  changes  may  be  needed  to  the  WAAS 
design  to  achieve  precision  approaches  down  to  200  feet  above  touchdown. 

The  difficulties  in  proving  the  integrity  requirement  have  occurred  largely 
because  FAA  management  and  the  integrated  product  team  underestimated 
the  complexity  of  resolving  the  integrity  issue  and,  as  a  result,  failed  to 
recognize  the  seriousness  of  the  problem.  Moreover,  FAA  did  not  closely 
monitor  the  contractor’s  effort  to  demonstrate  integrity,  and  members  of 
the  team  did  not  have  a  clear  understanding  of  their  roles.  Consequently, 
team  members  did  not  effectively  communicate  with  each  other  and  the 
contractor.  Lack  of  monitoring  and  poor  communications  have  been 
recurring  problems  in  FAA’s  air  traffic  control  modernization  program.  For 
example,  in  1996,  we  reported  that  inadequate  oversight  of  contractors’ 
performance  was  a  major  contributor  to  FAA’s  recurring  cost,  schedule, 
and  performance  problems  with  other  projects  in  the  modernization 
program.21 

According  to  a  WAAS  study  group  FAA  convened  in  October  1997,  it  would 
be  difficult  to  prove  WAAS’  stringent  requirements.  Therefore,  FAA  would 
need  both  a  sound  mathematical  approach  and  evidence  acquired  through 
operational  experience.22  This  group’s  conclusions  were  validated  in 
December  1997,  when  according  to  FAA,  it  discovered  at  a  key  project 
milestone  that  the  contractor  did  not  have  an  adequate  plan  for  proving  the 
WAAS  integrity  requirement.  Over  the  next  21  months,  FAA,  its  experts,  and 
the  prime  contractor  attempted  to  resolve  the  integrity  issue;  however,  not 
until  September  1999,  when  the  aircraft  certification  members  of  the 
integrated  product  team  became  actively  involved,  did  FAA  fully 
understand  the  difficulty  in  trying  to  prove  this  requirement.  We  found  that 
FAA’s  progress  in  resolving  this  issue  was  hampered  because  (1)  the 


21  See  Aviation  Acquisition:  A  Comprehensive  Strategy  is  Needed  for  Cultural  Change  at 
FAA  (GAO/RCED-96- 159,  Aug.  22,  1996). 

22  Final  Report  of  the  WAAS  Study  Group  (Oct.  16,  1997). 
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contractor  took  about  a  year  to  submit  the  limited  results  of  its  assessment 
of  the  integrity  issue  and  (2)  the  integrated  product  team  was  slow  to 
respond  to  advice  from  other  FAA  experts  that  the  contractor’s  integrity 
assessment  was  inadequate. 

FAA  officials  agreed  that  they  should  have  monitored  this  situation  more 
closely  but  provided  three  reasons  for  not  addressing  the  integrity 
problems  more  promptly  These  reasons  largely  concerned  the  actions  of 
FAA’s  senior  management  and  the  integrated  product  team  forWAAS: 

•  Competing  priorities  between  FAA’s  acquisition  and  aircraft  certification 
organizations,  which  are  part  of  the  integrated  product  team,  negated 
the  effectiveness  of  this  team’s  approach  for  meeting  the  agency’s  WAAS 
goals.  This  situation  may  have  developed  because  FAA’s  aircraft 
certification  organization  is  more  accustomed  to  being  involved  after  a 
project  is  developed,  rather  than  actively  participating  throughout  its 
development.  As  we  reported  in  1996,  FAA’s  product  teams  have  not 
always  forged  true  partnerships  across  organizational  “stovepipes.”23 

•  A  shortage  of  in-house  technical  expertise  and  the  team’s  attention  to 
other  important  issues,  such  as  systems  engineering,  prevented  the  team 
from  monitoring  the  situation  more  closely. 

•  FAA  did  not  have  a  sufficiently  defined  process  for  identifying  and 
conveying  to  the  contractor  the  results  that  would  be  acceptable  for 
proving  WAAS’  integrity. 

Moreover,  some  team  members  did  not  feel  empowered  to  take  actions  to 
address  performance  issues  because  they  did  not  believe  that  they  had 
senior  management's  support  in  dealing  with  contractor-related  problems. 
In  addition  to  these  internal  problems,  an  FAA  official  told  us  that  the 
contractor  lacked  sufficient  expertise  to  prove  the  integrity  requirement.  In 
any  case,  after  being  alerted  to  the  difficulty  in  proving  this  requirement  2 
years  ago,  FAA  is  only  now  beginning  to  pay  it  serious  attention. 


23  See  Aviation  Acquisition:  A  Comprehensive  Strategy  is  Needed  for  Cultural  Change  at 
FAA  (GAO/RCED-96- 1 59,  Aug.  22,  1996). 
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Underlying  these  problems  is  a  basic  issue  of  how  FAA  decided  to  proceed 
with  WAAS’  development.  Essentially,  FAA  took  an  acknowledged  high-risk 
approach:  It  agreed  on  a  design  for  the  system  and  established  milestones 
for  system  deployment  before  completing  the  research  and  development 
needed  to  demonstrate  the  system’s  capability.  IfFAA  continues  along  this 
path,  it  could  incur  significant  costs  for  other  system  components,  such  as 
satellites,  to  make  the  design  fully  operational — without  knowing  whether 
the  system  will  meet  its  performance  requirements.  A  FAA  senior  manager 
acknowledged  that,  in  hindsight,  the  agency  should  have  placed  more 
emphasis  on  how  it  would  prove  WAAS’  integrity  performance  prior  to 
agreeing  to  a  design  for  the  system.  As  we  have  reported  for  other  FAA 
modernization  projects,  when  the  agency  attempts  to  combine  different 
phases  of  system  development  in  an  effort  to  more  quickly  implement 
systems,  it  repeatedly  experiences  major  performance  shortfalls,  which 
lead  to  delays  and  additional  costs.24 

Recognizing  these  problems,  FAA  is  in  the  process  of  implementing  a  new 
approach  to  developing  WAAS.  Under  this  approach,  before  making 
additional  investments,  FAA  plans  to  allow  time  for  collecting  and 
evaluating  data  on  (1)  system  performance,  (2)  the  extent  to  which  users 
have  purchased  equipment,  and  (3)  the  availability  of  emerging  new 
technologies  for  the  new  navigation  system.  In  essence,  FAA  plans  to 
reevaluate  WAAS  at  critical  points — “checkpoints” — in  its  development. 

We  believe  that  reevaluating  WAAS  at  checkpoints  should  provide  critical 
information  for  deciding  about  the  need  for  future  investments.  Moreover, 
such  checkpoints  would  allow  FAA  to  better  assess  whether  it  should  shift 
resources  to  other  parts  of  the  navigation  system,  such  as  LAAS,  to  meet 
users’  needs.  While  FAA  recognizes  the  need  for  checkpoints,  it  has  not 
developed  a  detailed  plan  explaining  when  these  checkpoints  would  occur, 
what  they  would  accomplish,  and  who  would  be  responsible  for  overseeing 
them.  We  also  believe  that  it  would  be  prudent  for  FAA  to  have  progress  on 
WAAS  independently  validated  at  the  established  checkpoints  before  the 
Congress  approves  additional  funding. 


24  See  Air  Traffic  Control:  Observations  on  FAA’s  Air  Traffic  Control  Modernization  Program 
(GAO/T-RCED/AIMD-99- 1 37.  Mar.  25,  1999). 
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Software  Development 
Problems  Are  Likely  to 
Cause  Delays 


Given  past  problems  and  the  amount  of  and  potential  complexity 
associated  with  the  remaining  systems  engineering  development  effort — 
including  developing  algorithms  to  resolve  the  integrity  issue — we  believe 
that  the  WAAS  software  effort  will  continue  to  experience  delays.  For 
example,  the  schedule  for  the  initial  WAAS  capability  has  already  slipped 
14  months— from  July  1999  to  September  2000— largely  because  of 
problems  in  developing  the  WAAS  design,  which  led  to  more  software 
development.  At  the  end  of  its  initial  phase,  WAAS  will  have  approximately 
350,000  lines  of  code  and  370,000  additional  lines  or  more  of  code  will  be 
required  before  the  system  becomes  fully  operational.25  According  to  FAA, 
this  additional  code  will  be  needed  to,  among  other  things,  provide  for  the 
security  of  the  system  and  to  expand  its  operating  capability. 

We  have  reported  on  numerous  occasions  that  in  acquiring  software,  FAA 
has  not  adequately  detailed  its  design  requirements  and  overseen 
contractors’  development  of  the  software.26  Software  development— the 
most  critical  component  of  key  FAA  modernization  programs — has  been 
the  Achilles’  heel  of  FAA’s  efforts  to  deliver  programs  on  time  and  within 
budget. 


FAA  Underestimated 
Annual  Costs  for 
Maintaining  Software 


Even  if  all  navigation  system  and  development  issues  were  resolved,  the 
quantified  cost  of  the  new  navigation  system  is  likely  to  outweigh  the 
quantified  benefits  because  FAA  underestimated  the  cost  of  maintaining 
software  by  between  120  percent  to  over  200  percent,  according  to  our 
analysis.27  That  is,  we  found  that  FAA’s  estimate  of  $85  million  for  WAAS 
software  maintenance,  which  includes  the  staffing  cost  for  modifying  or 
adding  software  throughout  the  life  of  the  project,  was  understated  by 
between  $101  million  to  $181  million.  FAA’s  estimate  differed  from  our 
estimate  in  part  because  the  agency  used  more  optimistic  assumptions  in 
two  instances.  First,  FAA  assumed  that  7  percent  of  the  software  would  be 
changed  annually  throughout  the  life  of  the  project.  In  our  analysis, 
however,  we  assumed  that  10  percent  of  the  software  would  be  changed 


25  Software  size  is  usually  measured  in  lines  of  code.  A  line  of  code  is  a  set  of  instructions 
for  the  computer  to  perform  a  certain  task  and  is  one  basis  for  estimating  costs  for  items 
such  as  the  coding,  analysis,  design,  and  test  efforts  required  for  producing  the  line  of  code. 

20  See  Air  Traffic  Control:  Immature  Software  Acquisition  Processes  Increase  FAA  System 
Acquisition  Risks  (GAO/AIMD-97-47,  Mar.  21,  1997). 


1  This  assumes  no  value  to  any  passenger  time  savings. 
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annually.  We  made  this  assumption  because  (1)  FAA’s  own  data  suggested 
that  an  annual  factor  of  between  5  percent  to  10  percent  could  be  used  and 
(2)  the  model  we  used  suggested  that  for  software  projects  such  as  WAAS, 
annual  software  change  could  approach  10  to  20  percent.  Second,  FAA 
assumed  that  the  government  would  maintain  the  software  and  used  an 
hourly  labor  rate  of  $75  per  hour — the  rate  the  agency  uses  when  it 
maintains  software  in-house.  FAAs  own  data  suggested  that  having  the 
government  maintain  the  WAAS  software  would  be  optimistic,  given  its 
considerable  complexity  and  safety-of-life  purpose  and  the  government’s 
unfamiliarity  with  the  software’s  development.  For  our  analysis,  we  used 
the  prime  contractors  software  labor  rate  of  $127  per  hour  because  the 
contractor  is  very  familiar  with  the  software’s  development  and  well 
qualified  to  maintain  it. 


Two  Experts  in  Navigation 
Technology  Believe 
Separate  Equipment  Must 
Be  Used  in  Conjunction 


With  WAAS 


Costs  for  WAAS  may  also  be  higher  than  FAA  anticipates  if,  as  some 
navigation  experts  believe,  separate  equipment  is  needed  at  airports.  This 
equipment  would  be  used  to  monitor  the  information  WAAS  provides  to 
aircraft  as  they  approach  an  airport  for  a  landing.  Currently,  FAA  has  no 
plans  to  have  such  equipment  in  place.  But  these  two  experts  note  that  FAA 
has  always  ensured  safety  by  requiring  such  equipment  and  that  the  WAAS 
design  should  conform  to  this  requirement. 


To  illustrate  the  need  for  such  equipment,  these  experts  noted  that  local 
conditions  around  an  airport,  such  as  interference  from  man-made  and 
natural  occurrences,  may  affect  the  GPS  signal.  The  WAAS  reference 
station,  which  would  send  corrections  to  an  aircraft,  might  be  located 
hundreds  of  miles  from  the  airport  and  might  not  pick  up  these  local 
conditions.  Local  navigation  equipment  would  factor  in  local  conditions 
and  their  effect  on  an  aircraft’s  true  position.  Without  such  information, 
these  experts  maintain  that  a  pilot  will  not  have  the  critical  information 
needed  to  land  safely.  Even  though  this  occurrence  has  not  been  observed 
in  tests  to  date,  experts  contend  that  the  lack  of  observance  is  not  proof 
that  it  will  not  occur. 


In  response,  FAA  stated  the  agency’s  experts  have  reasoned  that  the  WAAS 
design  and  the  location  of  the  reference  stations  are  sufficient  to  account 
for  and  correct  any  inaccuracies  in  the  GPS  signal.  FAA  disagrees  that 
separate,  independent  monitoring  equipment  is  necessary  because,  given 
the  WAAS  design,  an  airport-based  monitor  will  not  provide  the  required 
integrity.  This  might  occur  because  an  airport  monitor  could  receive  its 
information  from  a  set  of  GPS  satellites  that  are  different  from  those  that 
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are  providing  information  to  the  aircraft.  Nevertheless,  FAA  asked  its 
experts  to  model  extreme  conditions,  and  these  experts  have  yet  to  detect 
local  inaccuracies.  FAA  also  notes  that  it  will  not  provide  guidance  for  the 
category  I  precision  approaches  until  it  has  had  an  opportunity  to  evaluate 
operational  data  and  feedback  from  pilots.  As  it  gains  experience,  FAA 
acknowledges  it  may  need  to  make  changes  to  WAAS,  which  could  increase 
the  cost  of  the  WAAS  investment. 


Other  Technologies  Do 
Not  Meet  FAA’s 
Requirements  and 
Users’  Needs,  but 
Experts  in  Competing 
Technologies  Contend 
That  Some  Needs  Are 
Overstated 


At  the  present  time,  alternative  technologies,  including  other  ground-based 
and  less  robust  satellite-based  systems,  fall  short  in  meeting  FAAs 
requirements  and  users’  needs.28  WAAS  is  designed  to  provide  precise 
position  and  landing  guidance  (category  I  precision  approaches)  to  meet 
FAAs  requirements  and  users’  needs  for  improved  safety  and  greater 
access  to  more  airports.  The  alternative  technologies  cannot  meet  FAA’s 
requirement  for  category  I  precision  approaches.  However,  according  to 
several  experts  supporting  alternative  technologies,  they  believed  users 
overstated  the  level  of  precision  needed  to  safely  land  at  more  airports  and 
therefore  a  less  robust  WAAS  or  other  technologies  could  satisfy  users’ 
needs.  However,  FAA  contends  that  WAAS  serves  other  purposes  in 
addition  to  serving  users’  needs.  FAA  told  us  that  other  modernization 
efforts  depend  on  the  precise  position  information  expected  from  WAAS 
and  that  if  the  agency  were  to  rely  on  technologies  other  than  WAAS,  it 
would  have  to  consider  redesigning  these  other  projects. 


Other  Technologies  Cannot 
Provide  Category  I 
Precision  Approaches 


Experts  on  technologies  that  compete  with  WAAS  acknowledge  that, 
unlike  WAAS,  these  technologies  cannot  provide  vertical  guidance  to  a 
height  of  not  less  than  200  feet  above  touchdown  (category  I  precision 
approaches)  at  more  airports.  Users— primarily  general  aviation  pilots — 
told  FAA  that  they  need  this  level  of  precision.  It  is  important  to  note  that 
FAA  has  yet  to  demonstrate  if  and  when  WAAS  will  provide  users  with 
guidance  for  category  I  precision  approaches. 


28  In  considering  FAA  requirements  and  users’  needs,  it  is  important  to  factor  in  the  cost  of 
providing  precision  landing  guidance.  FAA  notes  that  it  could  provide  this  guidance  through 
technologies  such  as  instrument  landing  systems  at  all  runway  ends  in  the  United  States,  but 
this  would  be  cost  prohibitive. 
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While  recognizing  that  other  technologies  cannot  provide  category  I 
precision  approach  guidance,  these  experts  question  whether  such 
guidance  is  needed.  They  note  that  providing  users  with  access  to  more 
airports  depends  upon  factors  other  than  the  precise  position  information 
WAAS  offers.  For  example,  to  experience  the  full  benefits  of  WAAS, 
airports  would  need  to  invest  in  infrastructure,  such  as  lighting  systems, 
and  pilots  would  need  training  to  fly  precision  approaches  into  these 
airports.  Given  this  potential  investment  and  other  considerations,  these 
experts,  who  support  technologies  that  compete  with  WAAS,  pointed  out 
three  different  available  combinations  of  ground-based  and  satellite-based 
technologies  that  could  meet  the  vast  majority  of  users’  needs  for  precise 
landing  information  at  a  lower  cost  than  WAAS:  (1)  Long  Range  Navigation 
(LORAN  C),  a  ground-based  navigation  system  that  provides  position  and 
timing  information  to  both  aviation  and  nonaviation  users;29  (2)  a  scaled- 
back  WAAS,  which  would  not  provide  category  I  precision  approaches;  and 
(3)  LORAN  C  with  an  enhancement  known  as  Eurofix.  For  example, 
according  to  a  1998  study  conducted  for  the  Department  of  Transportation, 
the  cost  to  develop  and  operate  LORAN  C  is  estimated  at  over  $600  million 
for  2000  through  20 15. 30  Similarly,  the  cost  to  develop  and  operate  WAAS 
during  this  period  is  estimated  at  $2.3  billion.  Most  of  the  difference  in  cost 
is  attributable  to  the  fact  that  WAAS  would  rely  on  newly  launched 
geostationary  satellites,  which  LORAN  C  does  not  need. 

These  experts  maintain  that  the  competing  technologies,  combined  with 
on-board  equipment  that  provides  an  increased  vertical  guidance 
capability,  can  guide  users  down  to  a  height  of  about  250  feet  and  therefore 
achieve  “near”  precision  approaches.31  They  therefore  contend  that  a  less 
costly  technology  than  the  WAAS  now  contemplated  could  still  meet  the 
vast  majority  of  the  aviation  community’s  needs  for  precise  landing 
information — especially  for  general  aviation  users,  who  are  expected  to 
benefit  the  most  from  WAAS.  At  runways  where  a  200-foot  minimum 


29  LORAN  C  was  originally  designed  as  a  primary  navigation  system  for  the  maritime 
community.  This  technology  is  ground-based  and  can  provide  an  independent  navigation 
service  that  does  not  rely  on  GPS. 

30  An  Assessment  of  the  Proposed  Phase  Out  of  the  LORAN  C  Navigation  System,  Booz-Allen 
&  Hamilton,  Inc.,  July  17,  1998. 

31  Users  can  receive  vertical  guidance  from  different  types  of  equipment.  For  example,  some 
commercial  aircraft  are  currently  equipped  with  flight  management  systems  that  are 
coupled  with  altimeters  to  provide  vertical  guidance.  Using  this  equipment,  pilots  can  make 
“near"  precision  approaches. 
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category  I  approach  is  critical,  these  experts  stated  that  these  requirements 
could  be  satisfied  by  LAAS. 

According  to  FAA,  it  analyzed  the  technical  capability  of  these  three 
alternatives  as  part  of  its  1999  reevaluation  of  its  plan  for  a  new  navigation 
system.  It  rejected  the  LORAN  C  and  scaled-back  WAAS  alternatives 
because  they  did  not  provide  the  precision  navigation  guidance  expected 
from  WAAS.  The  agency  rejected  the  Eurofix  alternative  because  it  did  not 
meet  the  integrity  requirements  for  precision  approaches.  FAA  also 
believes  that  the  alternative  technologies  and  on-board  equipment  would 
bring  pilots  down  to  about  350  feet,  not  the  250  feet  the  experts  contend 
could  be  achieved.  Moreover,  implementing  these  technologies  would  shift 
a  large  cost  from  the  government  to  general  aviation  system  users — to 
purchase  additional  equipment  estimated  at  about  $1,000  to  $4,000  per 
aircraft — and  yet  these  users  would  still  be  limited  in  their  ability  to  land  at 
more  airports  in  foul  weather.  A  group  representing  major  users  reaffirmed 
its  support  for  WAAS,  with  the  expectation  that  it  will  provide  category  I 
precision  guidance.  In  addition,  according  to  an  interest  group  representing 
general  aviation  aircraft  owners  and  pilots,  these  individuals  support  FAA’s 
approach  because  while  proponents  of  the  three  alternative  technologies 
make  claims  about  their  capabilities,  these  capabilities  have  yet  to  be 
validated.  Furthermore,  owners  and  pilots  are  not  in  favor  of  changing  the 
design  of  the  new  navigation  system  after  FAA  and  aviation  manufacturers 
have  agreed  to  specifications  for  equipment  design  and  invested  hundreds 
of  millions  of  dollars  in  this  new  design. 


Using  Other  Technologies 
Could  Require  Design 
Changes  in  Modernization 
Projects  That  Depend  on 
Precise  Position 
Information 


As  currently  designed,  several  of  FAA’s  airspace  modernization  projects 
depend  on  WAAS  for  precise  position  information  and,  according  to  FAA, 
could  not  readily  be  developed  without  WAAS.  While  GPS  provides 
position  information,  WAAS  is  designed  to  ensure  the  integrity  of  that 
information.  If  an  alternative  technology  is  used,  these  programs  may  need 
to  be  redesigned.  Chief  among  these  is  the  Automatic  Dependent 
Surveillance  Broadcast,  one  of  a  group  of  technologies  designed  to  allow 
FAA  to  implement  a  new  system  of  traffic  management.  Under  this  system, 
known  as  free  flight,  pilots  and  air  traffic  controllers  would  receive  more 
precise  information  about  the  location  of  aircraft  in  order  to  improve 
system  safety  while  allowing  airspace  and  airport  resources  to  be  used 
more  efficiently.  Specifically,  the  Automatic  Dependent  Surveillance 
Broadcast  communicates  information  about  an  aircraft’s  position  from  on¬ 
board  equipment  that  receives  signals  from  global  satellites  and  sends  this 
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information  directly  to  ground  receivers  and  to  nearby  aircraft.  With  such 
information,  more  aircraft  can  fly  with  increased  efficiency  and  safety. 

Other  projects  that  depend  on  precise  position  information  include 
enhanced  systems  to  warn  pilots  and  air  traffic  controllers  of  proximity  to 
the  ground  and  related  obstacles.  Depending  on  the  application,  the 
effectiveness  of  these  projects  might  be  somewhat  reduced  in  the  absence 
of  WAAS.  LAAS  could  provide  monitoring  ifWAAS  was  not  deployed; 
however,  it  would  only  provide  for  such  monitoring  at  a  specific 
location/airport  and  would  not  provide  the  coverage  expected  from  WAAS. 


Conclusions 


FAA’s  recently  announced  delays  in  the  initial  deployment  of  WAAS, 
because  of  difficulties  in  proving  integrity,  are  likely  to  delay  the  benefits 
and  raise  the  cost  of  the  new  navigation  system.  These  delays  have  not 
changed  FAA’s  original  plans  for  WAAS,  which  were  based  on  users’  needs. 
However,  if  it  is  determined  that  WAAS  cannot  perform  as  intended,  FAA 
will  need  to  revisit  its  investment  in  the  new  navigation  system. 

With  regard  to  WAAS  integrity,  FAA’s  management  and  its  integrated 
product  team  should  have  addressed  these  concerns  in  a  more  timely 
manner.  FAA  recognizes  that  its  management  of  this  new  system, 
particularly  WAAS,  has  fallen  short,  but  its  plans  for  resolving  current 
problems,  including  the  integrity  requirement,  and  preventing  future  delays 
are  ambiguous.  FAA  talks  about  establishing  checkpoints  to  ensure 
progress  and  the  appropriate  use  of  funds,  but  it  has  not  developed  a  plan 
to  assure  the  Congress  that  these  checkpoints  are  in  place  and  are  likely  to 
function  appropriately. 

Moreover,  program  success  will  only  come  about  if  senior  FAA 
management  embraces  and  fully  supports  the  integrated  product  team 
concept  and  establishes  an  on-going  process  for  the  team  to  reach 
consensus  on  how  the  contractor  must  demonstrate  that  a  project  meets 
the  agency’s  performance  requirements  and  to  convey  this  information  to 
the  contractor.  Otherwise,  more  projects  may  experience  the  same 
problems  WAAS  encountered. 

Finally,  our  past  reviews  of  FAA’s  efforts  to  develop  systems  show  that  the 
agency  does  not  always  inform  the  Congress  in  a  timely  fashion  of 
problems  it  is  encountering  before  requesting  additional  funds.  Potential 
users  also  need  information  on  system  performance  so  that  they  can  make 
informed  decisions  on  the  merits  of  purchasing  equipment  to  use  with  the 
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new  navigation  system.  If  further  delays  occur,  these  users  may  not 
continue  to  support  FAA  in  the  development  of  this  new  system,  as  they 
now  do. 


Recommendations  To  enhance  FAAs  ability  to  develop  its  new  navigation  system  within 

budget  and  on  time  while  meeting  performance  requirements,  we 
recommend  that  the  Secretary  of  Transportation  direct  the  Administrator 
of  FAA  to  take  the  following  actions: 

•  Develop  a  comprehensive  plan  that  would  provide  the  framework  for 
the  agency’s  future  investments  in  its  new  navigation  system.  This  plan 
should  establish  future  checkpoints  at  which  FAA  would  determine 
whether  (1)  the  contractor’s  approach  for  meeting  performance 
requirements  conforms  with  the  agency’s  guidelines,  (2)  users’  needs 
have  changed,  and  (3)  other  technologies  have  matured  and  could  meet 
users’  needs  and  the  agency’s  requirements. 

•  Have  an  external  organization  evaluate  the  agency’s  progress  at  these 
checkpoints  and  include  the  results  of  this  evaluation  in  the  agency’s 
request  for  future  funding  of  the  navigation  system. 


Agency  Comments 


GAO  provided  the  departments  of  Transportation  and  Defense  and  the 
Satellite  Navigation  User  Group  with  a  draft  of  this  report  for  their  review 
and  comment. 

GAO  met  with  officials  from  the  Department  of  Transportation,  including 
the  Program  Manager,  Radionavigation  and  Positioning,  Office  of  the 
Assistant  Secretary  for  Transportation  Policy  and  FAA’s  Product  Team  Lead 
for  the  Global  Positioning  System.  These  officials  acknowledged  the 
problems  addressed  in  the  draft  report  concerning  the  Wide  Area 
Augmentation  System  and  agreed  with  the  conclusions  and 
recommendations.  However,  they  expressed  two  major  concerns.  First, 
these  officials  stated  that  our  characterization  of  the  increases  in  costs  for 
the  Wide  Area  Augmentation  System  as  “cost  growth”  was  misleading 
because  the  increases  are  attributable  to  cost  growth  as  well  as  other 
factors,  including  a  change  in  how  the  department  calculated  project  costs. 
The  officials  provided  revised  estimates  to  reflect  total  lifecycle  costs 
covering  a  20-year  period.  To  address  this  concern,  we  revised  the  draft 
report  to  clarify  that  the  cost  increases  were  due  to  growth  as  well  as  other 
factors,  such  as  additional  program  support  costs.  The  focus  of  our  review 
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was  system  development;  therefore,  we  also  clarified  our  draft  report  to 
indicate  that  the  cost  estimates  represented  development  costs  only. 

Second,  these  officials  expressed  the  view  that  our  discussion  of  the 
benefits  from  the  Wide  Area  Augmentation  System  implied  that  users 
would  only  benefit  if  they  were  able  to  do  the  more  stringent  precision 
approaches  and  that  airports  would  need  to  spend  a  minimum  of  $1  million 
in  infrastructure  upgrades  to  get  precision  approach  capabilities  from  the 
Wide  Area  Augmentation  System.  These  officials  maintained  that  users 
would  receive  major  benefits  from  any  level  of  precision  guidance  into 
airports  and  that  airports  would  not  necessarily  need  to  upgrade  their 
facilities  to  provide  for  these  approaches.  We  agree  with  the  officials  that 
users  would  receive  benefits  even  from  the  limited  precision  approach 
capability  provide  by  this  system  and  that  airports  would  not  necessarily 
need  to  upgrade  their  investment  to  provide  for  these  approaches.  We 
revised  the  draft  report  accordingly.  These  officials  also  provided  us  with 
technical  clarifications,  which  we  incorporated  into  the  draft  report  as 
appropriate. 

The  Department  of  Defense  generally  concurred  with  the  draft  report.  In 
addition,  it  offered  two  comments.  First,  it  clarified  the  departments 
strategy  for  maintaining  the  Global  Positioning  System,  and  we  revised  the 
draft  report  accordingly.  Second,  it  commented  that  the  continued  delays  in 
the  Wide  Area  Augmentation  System  and  questions  about  worldwide 
interoperability  of  this  augmentation  system  could  create  additional  cost 
and  schedule  delays  and  adversely  impact  the  department’s  operations  if 
not  addressed.  (See  app.  IV  for  the  Department  of  Defense’s  comments.) 

GAO  met  with  representatives  from  the  Satellite  Navigation  User  Group, 
who  generally  agreed  with  the  draft  report.  These  representatives 
reiterated  their  support  for  FAA’s  efforts  to  field  a  new  navigation  system. 
In  light  of  this,  they  were  concerned  that  the  draft  report  did  not  recognize 
that  users  would  receive  benefits  even  from  the  limited  precision  approach 
capability  provided  by  the  Wide  Area  Augmentation  System  and  that 
airports  would  not  necessarily  need  to  upgrade  their  investment  to  provide 
for  these  approaches.  We  agreed  and  revised  the  draft  report  accordingly. 
These  officials  also  provided  us  with  technical  clarifications,  which  we 
incorporated  into  the  draft  report  as  appropriate. 


We  are  sending  copies  of  this  report  to  interested  Members  of  Congress; 
the  Honorable  Rodney  E.  Slater,  Secretary  of  Transportation;  the 
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Honorable  William  S.  Cohen,  Secretary  of  Defense;  and  the  Honorable  Jane 
F.  Garvey,  Administrator,  Federal  Aviation  Administration.  We  will  also 
make  copies  available  upon  request. 

If  you  have  any  questions  about  this  report,  please  contact  me  at  (202)  512- 
2834.  Key  contributors  are  listed  in  appendix  V. 


Sincerely  yours, 


It  PQ  * 


Gerald  L.  Dillingham,  Ph.D. 
Associate  Director, 
Transportation  Issues 
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Objectives,  Scope,  and  Methodology 


In  light  of  the  expected  cost  of  the  Federal  Aviation  Administration’s  (FAA) 
new  navigation  system  and  continuing  concerns  about  the  ability  of  the 
largest  component  of  that  system— the  Wide  Area  Augmentation  System 
(WAAS) — to  achieve  cost,  schedule,  and  performance  goals,  the  Chairman, 
Subcommittee  on  Transportation,  Senate  Committee  on  Appropriations, 
asked  us  to  provide  information  on  whether  (1)  the  Department  of 
Defense’s  (DOD)  current  Global  Positioning  System  (GPS)  or  its  planned 
improvements  for  GPS  can  meet  FAA’s  navigation  requirements,  (2)  the 
benefits  of  FAA’s  chosen  approach  to  an  augmented  system  currently 
outweigh  the  cost  of  this  system,  and  (3)  other  technologies  are  available  to 
meet  FAA’s  requirements  and  users’  needs  for  a  new  navigation  system. 

To  address  the  first  objective,  we  consulted  with  DOD  officials  and  GPS 
experts.  In  addition,  we  reviewed  a  1999  study  prepared  by  Johns  Hopkins 
University’s  Applied  Physics  Laboratory,  sponsored  in  part  by  FAA,1  that 
assessed,  among  other  things,  whether  GPS  can  meet  navigation 
requirements  for  civil  aviation.  In  addition,  we  reviewed  a  1998  study  by  the 
Mitre  Corporation,  which  supports  FAA  in  developing  the  national  airspace 
system,  that  analyzed  GPS’  current  and  future  ability  to  meet  navigation 
requirements. 

To  respond  to  the  second  objective,  we  discussed  with  FAA  and  the  Mitre 
Corporation,  the  process  they  followed  to  develop  a  range  of  alternatives. 
We  reviewed  guidance  from  the  Office  of  Management  and  Budget  (OMB) 
to  identify  analytical  issues  agencies  should  address  when  doing  benefit- 
cost  analyses.  We  also  reviewed  FAA’s  analysis  of  the  costs  and  benefits  of 
various  technologies  to  determine  whether  the  agency  followed  OMB 
guidance  and  used  empirical  data  to  support  its  findings.  In  our  review,  we 
also  confirmed  the  reasonableness  of  the  largest  costs  estimates — for 
equipment  on  board  the  aircraft  and  additional  satellites — with  equipment 
manufacturers,  satellite  providers,  and  aviation  industry  trade  associations 
and  special  groups,  including  the  General  Aviation  Manufacturers 
Association,  Aircraft  Owners  and  Pilots  Association,  the  Air  Transport 
Association  of  America,  and  the  Regional  Airline  Association.  The  latter 
four  associations  and  other  users  are  also  part  of  the  Satellite  Navigation 
User  Group,  which  was  created  to  achieve  a  consensus  in  the  user 


1  The  Johns  Hopkins  University  study  was  performed  under  contract  for  the  Air  Transport 
Association  of  America  (ATA)  and  jointly  sponsored  by  ATA,  FAA,  and  the  Aircraft  Owners 
and  Pilots  Association. 
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community  and  in  FAA  for  making  the  transition  to  the  new  navigation 
system. 

We  also  interviewed  representatives  from  the  Air  Line  Pilots  Association 
International  and  the  National  Air  Traffic  Controllers  Association  to  obtain 
their  views  on  the  reasonableness  of  cost  and  benefit  assumptions  made  in 
FAA’s  analysis.  Furthermore,  we  relied  on  our  past  work  related  to  WAAS  to 
compare  the  process  FAA  used  for,  and  the  results  of,  its  most  recent 
benefit-cost  analysis  with  a  similar  analysis  performed  in  early  1998.  While 
we  did  not  extensively  review  the  model  FAA  used  to  calculate  net  benefits 
and  benefit-cost  ratios,  the  methodology  of  this  model,  including 
spreadsheet  analysis  and  Monte  Carlo  Simulation,  is  commonly  used  in 
making  such  calculations.  Finally,  we  interviewed  representatives  from 
Raytheon,  the  prime  contractor  for  WAAS,  to  obtain  data  on  the  nature  and 
extent  of  problems  in  demonstrating  that  WAAS  can  meet  its  integrity 
requirement  and  in  developing  software  for  navigation.  To  determine 
whether  FAAs  estimates  of  the  costs  of  software  development  and 
maintenance  were  reasonable,  we  performed  an  independent  assessment 
of  the  contractor’s  cost  estimates  using  a  model  known  as  “CostXpert,” 
which  is  used  in  the  federal  systems  acquisition  community  to,  among 
other  things,  determine  the  costs  of  developing  projects  using  historical 
costs  of  completed  projects  with  software  development  factors  similar  to 
WAAS. 

Finally,  for  the  third  objective,  we  elicited  the  views  of  experts  in  aviation 
navigation  and  related  technologies  to  discuss  the  capability  of  other 
technologies  to  meet  FAA’s  requirements  and  users’  needs. To  accomplish 
this  effort,  we  provided  these  experts  with  detailed  data  on  the  alternative 
technologies  developed  by  the  Mitre  Corporation  that  served  as  the  basis 
for  FAA’s  decision  to  continue  with  the  new  navigation  system.  We  asked 
these  experts  a  series  of  questions  that  focused  on  whether  FAA  adequately 
considered  a  full  range  of  alternatives  and  whether  they  agreed  with  FAA 
when  the  agency  rejected  certain  technologies  from  its  new  navigation 
system.  Some  of  these  experts  supported  WAAS,  while  others  supported 
technologies  that  compete  with  WAAS.  Therefore,  where  we  identified 
conflicting  views  between  the  FAA  and  these  experts,  we  discussed  the 
points  of  contention  openly  with  both  sides  to  fully  understand  their 
positions,  and  we  present  the  views  for  and  against  using  alternative 
technologies.  (See  app.  II  for  background  on  the  experts  we  contacted 
during  this  review.)  Finally,  we  interviewed  FAA  officials  to  discuss 
whether  interdependencies  exist  between  WAAS,  LAAS,  and  other 
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modernization  projects,  and  the  impact  of  these  other  efforts  ifWAAS  is 
not  deployed  as  scheduled  or  is  terminated. 

We  conducted  our  work  from  July  1999  through  May  2000  in  accordance 
with  generally  accepted  government  auditing  standards. 
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Mr.  John  M.  Beukers 

Founder,  Beukers  Laboratories,  Inc.  (today  known  as  Beukers 
Technologies,  Inc.),  a  company  specializing  in  the  implementation  and  use 
of  radionavigation  systems  since  1963.  Mr.  Beukers  is  a  director  of  the 
International  Loran  Association  and  the  International  Navigation 
Association  and  also  serves  as  a  consultant  to  the  U.S.  government,  the 
European  Union,  and  others  on  radionavigation  policy. 

Dr.  John  Diesel 

Chief  Scientist,  Litton  Aero  Products.  Litton  Aero  is  a  manufacturer  and 
supplier  of  inertial  navigation  and  global  positioning  systems  to  the 
aviation  industry.  Dr.  Diesel  has  analyzed  several  navigation  systems— 
including  GPS,  Inertial  Reference  System,  Loran,  VHF  Omni-directional 
Range/Distance  Measuring  Equipment,  and  Instrument  Landing  Systems. 

Mr.  Paul  R.  Drouilhet 

Consultant  to  the  Director  of  the  Massachusetts  Institute  of  Technology’s 
Lincoln  Laboratories,  where  he  oversaw  programs  in  surveillance  and 
control  technology  and  air  traffic  control.  Mr.  Drouilhet  is  also  the 
Chairman  of  FAA’s  Subcommittee  on  Air  Traffic  Services  and  is  an  airplane 
pilot. 

Dr.  Per  Enge 

Professor,  Stanford  University,  Department  of  Aeronautics  and 
Astronautics;  Co-Director  of  the  GPS  Research  Laboratoiy.  Dr.  Enge  is  also 
a  developer  of  the  prototype  system  and  signal  for  WAAS  and  a  researcher 
for  the  prototype  of  LAAS. 

Dr.  G.  Benjamin  Hocker 

Principal  Research  Fellow,  Honeywell  Technology  Center.  Currently,  Mr. 
Hocker  focuses  on  the  evaluation  of  technologies  that  include,  among 
other  things,  inertial  navigation  systems. 

Mr.  George  H.  Quinn 

Until  1994,  Mr.  Quinn  was  FAA’s  National  Project  Manager  for  the 
establishment  of  the  LORAN  C  navigation  system  for  aviation.  Currently, 
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Mr.  Quinn  is  a  self-employed  engineering  consultant,  developing  ways  to 
use  GPS  in  vessel  tracking  systems. 

Mr.  William  Roland 

Private  consultant.  Former  Loran  C  Branch  Chief  and  Commanding  Officer 
of  the  U.S.  Coast  Guard’s  New  Jersey  Electronics  Engineering  Center.  Mr. 
Roland  recently  retired  from  his  position  as  President  of  Megapulse,  the 
company  responsible  for  the  development  and  construction  of  a  large  part 
of  the  world’s  Loran  transmitters. 

Dr.  G.  Linn  Roth 

President,  International  Loran  Association.  Dr.  Roth  is  also  the  president  of 
Locus,  Inc.,  a  developer  and  manufacturer  of  spread-spectrum  radio 
modules  for  integration  into  industrial,  utility,  GPS,  and  high-performance 
digital  Loran  receivers  for  navigation  and  timing  applications. 

Mr.  Robert  Siegfried 

Former  commercial  airline  pilot  with  approximately  24,000  hours  of 
experience.  Mr.  Siegfried’s  general  aviation  experience  totals 
approximately  12,000  hours — including  about  1,200  hours  operating 
helicopters. 

Mr.  Victor  Strachan 

Director,  Strategic  Development,  Litton  Aero  Products.  Mr.  Strachan  served 
as  a  navigator  in  the  Royal  Air  Force  for  18  years,  including  5  years  in  flight 
testing.  Mr.  Strachan  is  a  director  of  the  International  Navigation 
Association  and  a  member  of  the  Civil  Aviation  Council  of  the  Aerospace 
Industries  Association.  Litton  Aero  is  a  manufacturer  and  supplier  of 
inertial  navigation  and  global  positioning  systems  to  the  aviation  industry. 
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In  1999,  in  response  to  congressional  concerns  in  late  1998  about  its 
proposed  investment  in  WAAS  and  LAAS  and  the  vulnerabilities  associated 
with  satellite  navigation,  FAA  reevaluated  whether  the  investment  was 
sound.  As  a  first  step  in  this  reevaluation,  FAA  identified  four  alternatives 
that  combined  a  range  of  technologies,  based  on  different  levels  of 
investment.1  Three  of  the  four  alternatives  contained  one  or  more 
variations  of  WAAS  and  LAAS  as  well  as  variations  of  ground-based 
navigation  aids  that  could  serve  as  the  principal  backup  system.  The  four 
alternatives  were  the  following: 

•  Alternative  I:  No  WAAS  or  LAAS.  FAA  retains  its  existing  ground- 
based  navigation  infrastructure  and  enhances  it  to  accommodate 
demand. 

•  Alternative  II:  Simplified  WAAS  With  LAAS.  FAA  develops  a 
simplified  WAAS,  without  incorporating  precision  approach  guidance, 
develops  LAAS,  and  retains  about  50  percent  of  the  existing  ground- 
based  infrastructure  used  for  navigation  and  about  30  percent  of  the 
ground-based  infrastructure  used  for  precision  approaches. 

•  Alternative  III:  Less  Robust  WAAS  With  LAAS.  FAA  develops 
WAAS  with  precision  approach  capability,  including  LAAS,  for  more 
stringent  approach  requirements,  and  retains  about  50  percent  of  its 
ground-based  infrastructure. 

•  Alternative  IV:  Full  WAAS  With  LAAS.  FAA  develops  WAAS  with 
precision  approach  capability,  including  LAAS,  for  more  stringent 
approach  requirements,  and  retains  about  30  percent  of  its  existing 
ground-based  infrastructure. 

In  developing  the  alternatives,  FAA  made  three  major  assumptions.  First,  it 
assumed  that  the  new  navigation  system  would  not  be  the  only  system  that 
pilots  would  use  to  navigate  and  land — a  “sole-means”  system — as 
originally  intended.  By  moving  away  from  this  assumption,  FAA  realized 
that  it  would  need  to  retain  a  backup  system.  Second,  for  all  the 
alternatives,  FAA  assumed  that  all  new  commercial  jets  would  be  produced 
with  avionics  on  board  the  aircraft  to  help  pilots  navigate  (inertial 
navigation  systems),  which  would  be  used  in  conjunction  with  other 
ground-based  navigation  aids,  and  that  affordable  inertial  systems  would 
not  be  available  until  about  2008.  Finally,  FAA  assumed  that  the  Long  Range 
Navigation  (LORAN  C)  technology  would  continue  to  be  provided  and  used 
by  some  general  aviation  pilots  until  2008. 


1  In  total,  FAA  considered  12  alternatives  for  its  new  navigation  system. 
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After  identifying  the  four  alternatives,  FAA  analyzed  the  costs  and  benefits 
of  each.  According  to  this  analysis,  continuing  with  the  WAAS/LAAS 
system,  coupled  with  using  about  30  percent  of  its  current  ground-based 
infrastructure  as  a  backup  to  GPS,  would  yield  the  greatest  net  quantified 
benefits  (dollar  value  of  benefits  minus  costs),2  regardless  of  whether 
passengers’  time  savings  are  included.3,4  According  to  FAA,  over  the  period 
2000  through  2020,  there  is  an  80-percent  chance  that  the  net  benefits  of  its 
preferred  alternative  is  about  $2.5  billion  if  passengers’  time  savings  are 
counted  and  $72  million  if  these  savings  are  not  counted.  The  benefit-cost 
analysis  also  identified  at  least  1 1  benefits  that  were  not  quantified. 
Included  among  these  were  safety  benefits,  such  as  improved  surface 
surveillance,  as  well  as  operational  benefits,  such  as  enabling  more 
landings  at  airports  where  today’s  operations  are  limited  by  lack  of  ground- 
based  navigational  aids.  According  to  FAA,  its  decision  to  pursue  this 
approach  had  the  support  of  all  major  user  groups.5 

Table  2  summarizes  FAA’s  analysis  for  the  four  basic  alternatives  the 
agency  considered  in  terms  of  each  alternative  s  benefit-cost  ratio  and  net 
benefits,  expressed  as  present  values.6  FAA  considered  these  alternatives 
with  and  without  passengers’  value  of  time.  As  the  table  shows,  the  fourth 
alternative — which  is  the  one  FAA  decided  to  pursue — provided  a  benefit- 
cost  ratio  of  2.4,  when  passengers’  value  of  time  are  included,  and  a  ratio  of 
1.1,  when  these  values  are  not  included. 


2As  an  alternative  to  the  benefit-cost  ratio,  for  which  the  present  value  of  benefits  is  divided 
by  the  present  value  of  costs,  analysts  sometimes  calculate  the  present  value  of  net  benefits. 
This  value  is  equal  to  the  present  value  of  benefits  minus  the  present  value  of  costs. 

3  The  Department  of  Transportation  values  passenger  time  from  $22  to  $33  per  hour, 
depending  on  the  nature  of  air  travel.  In  1998,  we  reviewed  the  economic  literature  and 
found  that  no  consensus  exists  as  to  the  validity  of  using  small  increments  of  time,  in  part 
because  passengers  might  not  perceive  and  value  time  savings  of  as  little  as  30  seconds. 

4  It  should  be  noted  that  the  other  three  options  had  benefit-cost  ratios  of  less  than  1. 

5  The  Satellite  Navigation  User  Group  was  created  to  achieve  a  consensus  throughout  the 
user  community  and  with  FAA  for  making  the  transition  to  the  new  navigation  system. 

6  Since  benefits  are  divided  by  costs,  any  benefit-cost  ratio  above  1.0  indicates  that  the 
project  has  benefits  exceeding  costs,  and  any  ratio  below  1.0  indicates  that  the  project  has 
benefits  less  than  costs. 
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Table  2:  Summary  of  FAA’s  Economic  Analysis  of  Four  Alternatives 

Dollars  in  millions 

With  passenger  value  of 
time 

Without  passenger  value  of 
time 

Alternatives 

Benefit-cost 

ratio 

Net  benefits 

Benefit-cost 

ratio 

Net  benefits 

1.  No  WAAS;  No  LAAS 

1.5 

$280 

0.5 

($287) 

II.  Simplified  WAAS,  With 
LAAS 

1.0 

$94 

0.5 

($1,007) 

III.  Less  Robust  WAAS, 
with  LAAS 

2.1 

$1,857 

0.9 

($25) 

IV.  Robust  WAAS  with 
LAAS  While  Retaining  30 
Percent  of  Ground 
Infrastructure 

2.4 

$2,469 

1.1 

$72 

Note:  Benefit-cost  ratios  and  net  benefits  are  calculated  in  present  value  terms,  which  account  for 
future  benefits  and  costs,  here  expressed  in  1999  dollars. 

Source:  FAA. 


We  found  that  FAA  combined  empirical  data,  where  available,  with 
professional  judgment  to  develop  its  estimates  of  costs  and  benefits.  For 
example,  FAA’s  estimates  for  key  cost  elements,  such  as  geostationary 
satellites,  omboard  aircraft  equipment,  and  ground-based  navigation  aids, 
were  based  on  vendor  quotes  and  manufacturer  pricing  surveys.  We 
independently  validated  FAA’s  cost  for  developing  WAAS  software  and 
found  that  it  was  within  an  allowable  range.  Moreover,  we  found  that  FAA 
did  not  assume  that  some  benefits,  such  as  direct  routing — which  yields 
savings  from  shorter  flight  times — could  only  be  achieved  with  WAAS  and 
did  not  fully  attribute  these  benefits  to  WAAS  as  it  had  in  earlier  studies. 

Furthermore,  in  accordance  with  OMB  guidance  and  acceptable  practice, 
FAA  identified  a  number  of  benefits  that  could  not  be  quantified  because  of 
the  lack  of  verifiable  data  but  that  are  nonetheless  important  for 
decisionmakers  to  consider  when  choosing  an  investment  option.  For 
example,  OMB  guidance  notes  that  a  comprehensive  evaluation  of  the 
different  types  of  benefits  and  costs,  quantified  or  not,  can  be  helpful  in 
identifying  the  full  range  of  program  effects.  We  have  also  reported  that 
estimates  of  benefits  are  typically  uncertain  because  of  imprecision  in  both 
underlying  data  and  modeling  assumptions,  and  it  is  appropriate  to 
consider  nonquantified  benefits  when  choosing  investment  options. 
Industry  representatives  share  FAA’s  views  on  the  nonquantified  benefits 
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the  agency  has  identified,  and  some  believe  that  these  and  other 
nonaviation  benefits  could  be  substantial  if  they  could  be  quantified. 

Despite  these  strengths,  we  found  that  one  key  benefit — FAA’s  estimate  of 
the  $4.4  billion  in  savings  achieved  by  allowing  plahes  to  decrease  the  time 
needed  for  an  approach  to  landing — was  based  largely  on  professional 
judgment,  without  empirical  data.  This  benefit  accounted  for  nearly  42 
percent  of  all  quantified  benefits.  Specifically,  FAA  assumed  that  40  percent 
of  all  flights  during  nonpeak  periods  could  decrease  approach  time  into 
airports,  even  though  it  recognized  that  the  percentage  could  range  from  20 
percent  to  50  percent  during  nonpeak  periods.  Because  of  the  lack  of 
empirical  data,  we  asked  FAA  to  perform  a  more  conservative  analysis, 
assuming  that  only  20  percent  of  the  flights  would  benefit.  We  found  that 
this  resulted  in  a  benefit-cost  ratio  of  1.9  if  passenger  value  of  time  was 
counted  and  a  ratio  of  .8  if  not  counted. 
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OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 
6000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-6000 

May  25,  2000 


COMMAND,  CONTROL, 
COMMUNICATIONS.  ANO 
INTELLIGENCE 


Dr.  Gerald  L.  Dillingham 
Associate  Director,  Transportation  Issues 
Resources,  Community,  and  Economic 
Development  Division 
U.S.  General  Accounting  Office 
Washington,  D.C.  20548 

Dear  Dr.  Dillingham: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  General  Accounting  Office 
(GAO)  draft  report,  National  Airspace  System:  Persistent  Problems  in  PAA’s  New  Navigation 
System  Highlight  Need  for  Periodic  Reevaluation”,  dated  May  9,  2000  (GAO  Code  3481 84/OSD 
Case  1999).  The  DoD  concurs  with  comments  on  the  draft  report.  A  comment  is  provided  to 
improve  the  technical  accuracy  of  the  report  and  a  second  comment  expresses  concern  with  the 
program  delay  and  worldwide  interoperability. 

In  the  second  paragraph  of  the  report  in  the  “Background”  section,  page  4,  the  following 
statement  is  made:  “The  DoD  operates  27  orbiting  GPS  satellites  to  ensure  a  basic  configuration 
of  24  working  satellites  at  any  given  time  for  both  military  and  civilian  use.”  A  much  more 
accurate  statement  of  the  GPS  orbital  maintenance  philosophy  is  contained  in  the  1999  Federal 
Radionavigation  Plan  (FRP),  paragraph  3.2. IB,  which  states:  “The  GPS  constellation  is 
configured  and  operated  to  provide  the  SPS  signals  to  civil  users  in  accordance  with  the  GPS 
Signal  Specification.  The  DoD  will  maintain  a  24-satcIlite  constellation.  Replacement  satellites 
will  be  launched  on  an  expected  failure  strategy  (a  replacement  satellite  is  launched  when  there 
are  indications  that  a  satellite  should  be  replaced).”  Recommend  that  the  statement  from  the  FRP 
replace  the  above  quoted  statement  from  the  draft  report. 

As  a  National  Airspace  System  user  and  potential  user  of  the  WAAS  ranging  source 
integrity  and  accuracy  enhancements  for  precision  approach,  the  Department  is  concerned  with: 
(1)  the  continued  delays  of  the  WAAS  Program,  and  (2)  worldwide  interoperability  of  this  GPS 
augmentation.  Failure  to  address  these  concerns  could  create  additional  costs  and  schedule 
delays  for  implementation,  operational  impacts,  and  difficulty  in  flying  DoD  aircraft  in  foreign 
countries. 


The  Department  appreciates  the  opportunity  to  comment  on  the  draft  report. 


Sincerely, 


Robert  M.  Nulwell,  RADM,  USN 
Deputy  Assistant  Secretary  of  Defense 
(C3ISR  and  Space) 
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